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IMAGE REGISTRATION METHOD IMPROVEMENT 
Field of the Invention 

The present invention relates generally to image registration and, in particular, to 
determining a transformation that registers two images that are related by rotation, scale 
and translation. 

5 Background 

Image registration is the process of determining the correspondence between 
pixel elements in a pair of images that have common subject matter. In particular, image 
registration involves determining the parameters of a transformation that relates the pixel 
elements in the pair of images. Image registration is therefore an important aspect of 
10 image matching, where two images are compared for common subject matter under the 
assumption that some geometrical transformation exists that relates substantial portions of 
the two images. Image registration is also used in satellite and medical imagery where a 
series of partially overlapping images are obtained, and a mosaic of the individual images 
has to be formed to thereby form a single large image. 

Image registration is also useful in camera calibration, where an image of a 
known object is captured and the location of that known object within the image is 
calculated to determine some unknown parameters of the imaging device. Yet another 
application of image registration is as part of a system for determining the relative video 
camera orientation and position between video frames in a video sequence. 

A simple form of image registration may be used when the two images are 
related through translation only. In such a case, a matched filtering method may be used 
to find the translation relating the two'images. Cross-correlation and phase-correlation 
are examples of two such matched filtering methods. 
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Cross-coirelation may be performed in the spatial or frequency domain. 
Consider two images, /,(*,>>), and I 2 (x,y) that are functions of pixel coordinates x and 
y. The cross-correlation of the two images I t {x,y), and I 2 {x,y) in the spatial domain 
is defined by: 



a* , ./)-£E / .(*.^<*+* , o'+y) 



* y 



(i) 



If the two images /,(*,,), and I 2 (x,y) are related by a simple translation 
(& x ,& y ) whereby: 

• Atey) (2) 

s. * 

then the cross-correlation C(*«,/) has a maximum value at the translation 
10 coordinates (A^.aJ where: 



(3) 



Thus, by calculating the cross-correlation C(x',y<) of the two images I^y), 
and I 2 (x,y), the translation (A„aJ that registers the two images /,(*,;,), and /,(*,.,,) 
may be determined. 

15 Cross-correlation is generally performed using the Fast Fourier Transform 

(FFT). For an image /„ y) , the discrete Fourier transform 3(/) is defined by: 

3[/J(«,v) = f£ In(Xiy)e -zn* UINxe -™y*,» y 

JC-0 y=0 V ' 

where tf, and are the image dimensions in the x and y dimensions 
respectively. The inverse discrete Fourier transform 3"' (F) is defined by: 



630938.doc 



5 " (F) = lT^ttni}(u,v)e^e 2 ^' N ' . (5) 

The FFT is a computationally efficient method of calculating the Discrete 
Fourier Transform 3(7) and its inverse Z~\F). 

The cross-correlation C may be calculated through: 

C = 3-(3(7,)3(7 2 )'), (6) 

where 3(7 2 )* denotes the complex conjugation of the Discrete Fourier 
Transform 3(7 2 ). Thus, taking the inverse FFT 3"'0 of the product of the FFT of one 
image 3(7,) and the complex conjugate of the FFT of the other image 3(7 2 )* leads to a 
further image which contains the values of the cross-correlation C which is equivalent to 
that defined in Equation (1 ). 

Phase-correlation C is another matched filtering method that is often used and 
is defined by: 

Wll^ (7) 

That is, rather than using the discrete Fourier transforms 3(7) of the images 
T,(x,y) , and I 2 (x,y), only the complex phase part of the discrete Fourier transforms of 
the images are used. If the images I t (x,y) , and I 2 (x,y) are related by a translation^ 
offset (a„A,), the phase correlation C will have a very sharp peak at the translation 
coordinates (a,,A,) that relates the two images 7, (*„>,), and I 2 {x,y), and small values 
elsewhere in the correlation C . 

Whereas matched filtering is used when the two images I^x.y), and I 2 (x,y) 
are related through translation only, when the two images I t (x,y), and I 2 (x,y) are 
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related by a rotation and a scale transformation, such that image I 2 (x,y) is a rotated and 
scaled version of image I t (x,y), i.e. 

'2 (*>.)') = 1 1 (s(x cos 5 + y sin 5), s(-xsin S + ycos &)) , ■ (8) 

where * is a scale factor and £> is a rotation angle, the unknown rotation S and 
5 scale s parameters may be determined by transforming the images /, (x, y) , and I 2 (x, y) 
into a log-polar coordinate space through: 

p = Uog(x 2 +y 2 ) 

The translation above leads to a relationship between the images /,(x,y), and 
1 2 (*» y) > n the log-polar space is as follows: 

10 / 2 (A0) = /,(p + log^ + $), (10) 

The matched filtering methods described above may then be used to determine 
the scale and rotation parameters s and & from the peak in the correlation C at coordinate 
(log s, &). 

However, image registration is often applied to a pair of images /,(x,y), and 
I 2 (x,y) where the correspondence between pixel elements is not a simple 
transformation, such as a translation. For example, it may be necessary to register two 
images /, (x, y) , and I 2 (x, y) that are captured using different imaging devices, or by the 
same imaging device but where each image is captured using a different configuration of 
the imaging device. In such cases the transformation includes translation, rotation and 
20 scaling. 
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Consider two images /,(*, y), and f 2 ( x , y) that are related by a translation as 
well as a rotation and a scale, such that: 

hix,y) = I,(s(xcos9 + vsinS) + A x ,s(-xsin& + ycos&)+ A,) (1 1) 

Current methods of registering images related by translation, rotation and scale," 
suffer from poor signal to noise ratios over a large class of images when the 
transformation parameters are such that the overlap between the two images is 
significantly reduced. Furthermore, current methods contain a 180-degree ambiguity, 
which leads to computational inefficiencies. This ambiguity is the result of the fact that 
the Fourier magnitude of a real function is symmetric. 



10 



Summary 

It is an object of the present invention to substantially overcome, or at least 
ameliorate, one or more disadvantages of existing arrangements. 

According to a first aspect of the present disclosure, there is provided a method 
of determining at least rotation and scale parameters of a transformation relating two 
1 5 images, said method comprising the steps of: 

forming a spatial domain representation of each of said images that is invariant 
to translation of said image; 

performing Fourier-Mellin correlation between said representations; 
detecting a magnitude peak in said correlation; and 

20 determining said rotation and scale parameters from the position of said 

magnitude peak. 

According to a second aspect of the present disclosure, there is provided a 
method of determining at least rotation and scale parameters of a transformation relating 
two images, said method comprising the steps of: 
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forming a multi-channel function of each of said images by applying an operator 
to said images, said operator being commutative within a constant to rotation and scale; 

forming a representation of each of said multi-channel functions that is invariant 
to translation of said multi-channel function; 
5 performing Fourier-Mellin correlation between said representations; 

detecting a magnitude peak in said correlation; and 

determining said rotation and scale parameters from the position of said 
magnitude peak. 

According to another aspect of the present disclosure, there is provided an 
10 apparatus for implementing any one of the aforementioned methods. 

According to another aspect of the present disclosure there is provided a 
computer program product including a computer readable medium having recorded 
thereon a computer program for implementing any one of the methods described above. 
Other aspects of the invention are also disclosed. 

15 Brief Description of the Drawings 

One or more embodiments of the present invention will now be described with 
reference to the drawings, in which: 

Fig. 1 is a flow diagram of a method of determining a transformation relating 
two images according to an embodiment of the present invention, with the transformation 
20 including rotation, scale and translation; 

Fig. 2 is a schematic block diagram of a general purpose computer upon which 
arrangements described can be practiced; 

Fig. 3 shows a more detailed flow diagram of a step of determining the rotation 
and scale parameters which relates the two images, and performed in the method shown 
25 in Fig. 1; 
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Fig. 4 shows a more detailed flow diagram of a step of determining the 
translation parameters which relates the two images, and performed in the method shown 
in Fig. 1; . 

Figs. 5 and 6 show more detailed flow diagrams of alternate implementations of 
5 forming a complex image I n ( x ,y) from an image with real values, and are sub-steps of a 
step in Fig. 3; 

Fig. 7 shows a more detailed flow diagram of forming a representation that is 
translation invariant in the spatial domain of one of the complex images, and are sub-steps 
of a step in Fig. 3; and 

10 Fig. 8 shows a more detailed flow diagram of performing Fourier-Mellin 

correlation, which is performed in a step in Fig. 3. 

Detailed Description 

Where reference is made in any one or more of the accompanying drawings to 
steps and/or features, which have the same reference numerals, those steps and/or features 
15 have for the purposes of this description the same functions) or operation(s), unless the 
contrary intention appears. 

It is to be noted that the discussions contained in the "Background" section 
relating to current methods of registering images should not be interpreted as a 
representation by the present inventors or patent applicant that such methods in any way 
20 form part of the common general knowledge in the art. 

Fig. 1 is a flow diagram of a method 200 of determining a fransfoimation 
relating two images according to an embodiment of the present invention, with the 
transformation including rotation, scale and translation. The method 200 of determining a 
transformation relating two images is preferably practiced using a genenu-purpose 
25 computer system 100, such as that shown in Fig. 2 wherein the method of Fig. 1 may be 
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implemented as software, such as an application program, executing within the computer 
system 100. In particular, the steps of the method 200 of determining a transformation 
relating two images are effected by instructions in the software that are carried out by the 
computer system 100. The software may be stored in a computer readable medium, 
5 including the storage devices described below, for example. The software is loaded into 
the computer system 100 from the computer readable medium, andtiien executed by the 
computer system 100. A computer readable medium having such software or computer 
program recorded on it is a computer program product. The use of the computer program 
product in the computer system 100 preferably effects an advantageous apparatus for 
1 0 determining a transformation relating two images. 

The computer system 100 is formed by a computer module 101, input devices 
such as a keyboard 102 and mouse 103, output devices including a printer 115 and a 
display device 1 14. A Modulator-Demodulator (Modem) transceiver device 1 16 is used 
by the computer module 101 for communicating to and from a communications 
15 network 120, for example connectable via a telephone line 121 or other functional 
medium. The modem 1 16 can be used to obtain access to the Internet, and other network 
systems, such as a Local Area Network (LAN) or a Wide Area Network (WAN), and may 
be incorporated into the computer module 1 01 in some implementations. 

The computer module 101 typically includes at least one processor unit 105, and 
20 a memory unit 106, for example formed from semiconductor random access memory 
(RAM) and read only memory (ROM). The module 101 also includes an number of 
input/output (I/O) interfaces including a video interface 107 that couples to the video 
display 114, an I/O interface 113 for the keyboard 102 and mouse 103, and an 
interface 108 for the modem 1 16 and printer 1 15. A storage device 109 is provided and 
25 typically includes a hard disk drive 110 and a floppy disk drive 111. A CD-ROM 
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drive 1 12 is typically provided as a non-volatile source of data. The components 105 
to 1 13 of the computer module 101, typically communicate via an interconnected bus 104 
and in a manner which results in a conventional mode of operation of the computer 
system 100 known to those in the relevant art. 
5 Typically, the application program is resident on the hard disk drive 110 and 

read and controlled in its execution by the processor 105. Intermediate storage of the 
program and any data fetched from the network 120 may be accomplished using the 
semiconductor memory 106, possibly in concert with the hard disk drive 110. In some 
instances, the application program may be supplied to the user encoded on a CD-ROM or 
10 floppy disk and read via the corresponding drive 1 12 or 1 1 1, or alternatively may be read 
by the user from the network 120 via" the modem device 116. The term "computer 
readable medium" as used herein refers to any storage or transmission medium that 
participates in providing instructions and/or data to the computer system 100 for 
execution and/or processing. 

15 The present invention also implicitly discloses a computer program, in that it 

would be apparent to the person skilled in the art that the individual steps of the preferred 
method 100 described herein are able to be put into effect by computer code. 

A method is disclosed that may be used for determining a fransformation relating 
two images and I 2 (x,y) that are assumed to be related by rotation, scale and 

20 translation, such that: 

h (*>y) = I x {s(xcos& + ysmff) + & x ,s(-xsin9 + ycosQ) + A y ) (12) 
where s is a scale factor, S is a rotation angle, and (a„A„) is a translation. 
The method comprises two stages - a first which determines the unknown scale and 
rotation transformation up to a one-hundred and eighty degree ambiguity in the rotation 
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angle 9 , and a second which 'determines the translation (a„ A y ) and resolves the angut 
ambiguity. The Fourier transform of the scaled, rotated and shifted image I 2 (x,y) is 
related to the other image /, (x,y) through 



s 



(13) 



In the first stage, by taking the magnitude of the Fourier transform 3[/,], 
translation invariant of the image is formed, 



1^X^)1 =-L 
N 



5[/,]( 



KCosj9+vsin 9 -usin.9 + vcos9 



(14) 



The translation invariant is not dependent on the translation (A ,A ) of the 

x y / 

image. Performing a log-polar transformation of the Fourier magnitude leads to a simple 
linear relationship between the Fourier magnitudes of the two images as follows: 

Hh]{R,<l>)\ = ~\^I x ]{R~\ogs^ + ff)\ . (15) 

A correlation between a log-polar resampling of the Fourier magnitude of the 
two images contains a peak at fog* and 9, thereby allowing the unknown scale s and 
rotation angle 9 parameters relating the two images I t (x,y) and I 2 (x,y) to be 
determined, with the rotation angle 9 having a 180-degree ambiguity. This ambiguity is 
the result of the fact that the Fourier magnitude of a real function is symmetric. 

The second stage of the method of determining the transformation relating the 
two images I } {x,y) and I 2 (x,y) starts by undoing the now known scale and rotation 
translations for both possible rotation angles 9 for the second image I 2 (x,y) to 
produced a partially registered image. The partially registered image is then correlated 
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with first image I x (x,y) to determine the unknown translation (A,,*,) between the two 
images /,(*, y) and I 2 (x,y). The rotation angle & that gives the best spatial correlation 
between the partially registered image and the first image /,(*,,) is considered to be the 
correct rotation angle & . The complete transformation relating the two images I i(x ,y) 
5 and l 2 (x,y) is now known. 

Referring again to Fig. 1 where the flow diagram of method 200 of determining a 
transformation relating two images /,(*,,) and according to the present 

invention is shown. The method 200 receives in step 205 as input the two images 
/,(*,,) and I 2 ( x , y) . The images /,(x,y) and I 2 ( x ,y) are assumed to have a 
10 substantial overlap in image content. The images I l(x , y) and I 2 ( x , y) are functions 
with real values, which are typically represented by an array of values between 0 and a 
predetermined maximum value, commonly 1 or 255. The images /,(,.,) and I 2 ( x ,y) 
are typically, retrieved from the storage device 109 (Fig. 2) in step 205. However, the 
images /,(*,;,) and may ^ be receiyed fr<Jm ^ ^ ^ ^ ^ ^ 

15 imaging device (not illustrated) connected to the computer system 100. 

Image registration is next performed in steps 210 and 270. In particular, the 
rotation and scale parameters, S and , respectively, which relate the two images 
I,(x,y) and /,(*,,) are determined in step 210, and the translation (a„A„) is 
determined in step 270. Method 200 ends in step 290 where an output is produced to the 
20 display device 114 containing two substantially aligned images Ifoy) and I 2 (x,y). 
Alternatively the scale factor s, rotation angle &, and translation (A„A,) are output on 
the display device 1 14. 

Fig. 3 shows a more detailed flow diagram of step 210 (Fig. 1) where the rotation 
and scale parameters, & and * respectively, which relate the two images /,(*,,) and 
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I 2 (x,y) are determined. Step 210 of the preferred implementation starts in sub-step 212 
where the processor 105 (Fig. 2) forms a multi channel function from the images /,(*, y) 
and I 2 (x,y) . In the preferred implementation the processor 105 forms complex images 
I,(x,y) and I 2 (x,y) from the images /,(*,v), and I 2 (x,y), such that when each 
complex image I n (x,y) is Fourier transformed, a non-Hermitiah result with a non- 
symmetric Fourier magnitude is produced. Therefore, using a complex image J„(x,y) as 
the input to the Fourier-Mellin correlation that follows, a 180-degree ambiguity that 
would otherwise exist is removed. 

The complex images I x (x,y) and I 2 {x,y) are formed by applying an operator 
y{ } to the images Ifay) , and I 2 {x, y) , where the operator y{ } is commutative within 
a constant to rotation and scale, ie. 

i>, W{*,y)}]= t&,sW fi . lf(*,y)h (16) 

where p and i are rotation and scale factors, T Rs is a rotation-scale 
transformation, and g is some function of rotation >0and scale s. 
Examples of the operator y{ } include: 

r\f{*>y)} = f{x,y)+if 2 {x,y); (17) 
^)} = f-f;and (18) 

Preferred steps of forming a complex image /„(*, y) from an image I„(x,y) are 
described below with reference to Figs. 5 and 6. 
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The multi channel functions formed in sub-step 212, those being the complex 
images fay) and fay) in the preferred implementation, are then each processed by 
the processor 105 in sub-step 214 to form a representation fay) and fay) of each 
of the two complex images fay) and T 2 ( x ,y), where representations fay) and 
fay) are translation invariant in the spatial domain. It is understood that translations 
of images cause cropping of the images. Translation invariant therefore means 
substantially translation invariant, as the cropping introduces changes to the images 
separate to changes introduced by the translation. 

Sub-step 214 is followed by sub-step 216 where the processor 105 performs 
Fourier-Mellin correlation on the representations T t (x,y) and T 2 (x, y) of the two 
complex images fay) and fay), thereby producing a phase correlation image in 
which possible rotation and scaling that relate the input images fay) and fay) are 
represented by isolated peaks. Fourier-Mellin correlation is described in more detail 
below with reference to Fig. 8. 

Because the representations T t (x,y) and T 2 {x,y) are translation invariant in the 
spatial domain, the Fourier-Mellin correlation produces superior results for images 
fay) and fay) that are related by a wide range of translation, rotation and scale 
factors. Such superior results typically include increased matched filter signal to noise 
ratio for images that are related by rotation, scale and translation transformation, and 
enhanced discrimination between images that are not related by a rotation, scale and 
translation transformation. 

Method 200 continues to sub-step 218 where the processor 105 detects the 
location of a magnitude peak within the phase correlation image. The location of the. 
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magnitude peak is interpolated through quadratic fitting, thereby detecting the location of 
the magnitude peak to sub-pixel accuracy. 

The processor 105 then determines in sub-step 220 whether the detected 
magnitude peak has a signal to noise ratio that is greater than a predetermined threshold. 
5 The threshold used in the preferred implementation is 1 .5. 

If it is determined that the magnitude peak has a signal to noise ratio that is 
greater than the predetermined threshold, the processor 105 uses in sub-step 222 the 
location of the magnitude peak to determine the scale s and rotation angle 9 parameters 
which relate, the two images /,(*,,) and I 2 ( x , y) . If the magnitude peak is at location 
10 «T,«), then the scale , and rotation angle 3 parameters which relate the two images 
/,(*,;;). and h(x,y) are: 

•^"^and 

(20) 

^ 2na 

"~ Q ' (21) 

where a and Q are constants related to the log-polar resampling step 630 of the 
15 Fourier-Mellin correlation described below with reference to Fig. 8. 

If it is determined in sub-step 220 that the peak has a signal to noise ratio that is 
not greater than the predetermined threshold, then it is concluded that the images /,(*,,) 
and I 2 ( x , y) are not related by rotation and scale transformations, and the method 200 
ends in sub-step 250. 

Fig. 4 shows a more detailed flow diagram of step 270 (Fig. 1) where the 
translation (a„aJ, which relates the two images /,(*,,) and I 2 ( x , y) , is determined. 
In sub-step 280 the rotation and scale transformations, determined in sub-step 222 (Fig. 
3), are performed on image /,(*,,), thereby undoing those transformations to form 
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image I\{x, y) . Optionally the transformations may be performed on the complex image 
I,(x,y). The rotated and scaled transformed image I[{x,y) and the original image 
I 2 {*>y) are then correlated in sub-step 282, using phase correlation,. to produce another 
correlation image. The position of a magnitude peak in this correlation image will 
generally correspond to the translation (a„aJ relating images I^y), and I 2 (x,y). 

Accordingly, in sub-step 284 the processor 105 detects the location of the magnitude peak 
within the correlation image. 

The processor 105 then, in sub-step 286, uses the location of the magnitude peak 
to determine the translation parameters (a„A„) which relate the two images I[(x,y) and 
h(x,y). The same translation parameters (a„A,) also relate the original two images 
/,GW0 and I 2 (x,y). if the magnitude peak is at location (x 0 ,y 0 ), then the translation 
(A,, A,) is (-x 0 ,-y 0 ) . Thus, the unknown scale s and rotation angle & parameters have 
been determined in sub-step 222, and the unknown translation parameters (a„A,) have 

been determined in sub-step 286. Those parameters are passed to step 290 (Fig. 1) for 
producing the output. 

Fig. 5 shows a more detailed flow diagram of a first implementation of sub-step 
212 (Fig. 3) where the complex image I 0 ( x ,y) is formed from the image /„(*,,). fo 
sub-step 320 the image /„(,,,) is convolved with a complex kernel function k by the ' 
processor 105. The convolution may be performed in the spatial domain or through the 
standard technique of multiplication in the Fourier domain. The complex kernel function 
* used in sub-step 320 is that with a Fourier transform K = 5(k) of: 

AT(«,v)=ii±il. 

J« + /vJ (22) 
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An alternative complex kernel function k' that may be used in sub-step 320 is 
one with a Fourier transform K' = %{k') of: 

K'(u,v) = u + iv. (23) 

The result of the convolution ((/*Ar), where * . denotes convolution,) is 
normalised in sub-step 330 to have unit magnitude, 

I *k 

T = V*k\' ' (24) 

Finally the normalised result of the convolution T is multiplied with the original 
image I a (x,y) in sub-step 340 to form the complex image 7„(x,y). The complex image 
I„(x,y) has the same magnitude as the original image I n (x,y), but each point in the 
complex image /„(*, y) has an associated phase generated by the convolution in sub-step 
320. For the kernels k and k' given in Equations (22) and (23), the associated phase 
encodes a quantity related to the gradient direction of the image I„(x,y) . 

Fig. 6 shows a more detailed flow diagram of a second (alternate) 
implementation of sub-step 212 (Fig. 3) where the complex image T a (x,y) is formed 
from the image I tt (x,y) . In sub-step 420 the processor 105 applies a non-linear operator 
to the image I n (x,y) to produce a complex image. The non-linear operator applied in 
sub-step 420 is the energy operator, which may be described by 

E[I] = ID>I-(DI)\ (25) 

where D is the derivative operator 

JLs = • -u / 

dx dy- ■ (26) 
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An alternative non-linear operator that may be applied in sub-step 420 to 
produce the complex image is the uni-modular energy operator: 

E'[l]=ID*I-(D'iy, (27) 

where D' is the uni-modular derivative operator. The uni-modular derivative 
operator D' may be described as an operation in the Fourier domain as follows: 



*V)=3-.[ 3W £-]. 



(28) 



non- 



Preferably, in sub-step 430 which follows sub-step 420, the result of the 
linear operator applied to image I n (x,y) is normalised to unit modulus, and the result of 
this normalisation is multiplied by the original image /„(*, v) in sub-step 440 to form the 
B complex image I„(x,y) . Alternatively, the result of the non-linear operator applied to 
image I a (x,y), hence the output of sub-step 420 may be used as the complex image 
L(x,y). 

Fig. 7 shows a more detailed flow diagram of forming a representation T a (x t y) 
of one of the complex images /„(*, y) that is translation invariant in the spatial domain 
performed in sub-step 214 (Fig. 3). Sub-step 214 receives as input the complex images 
IMy) formed in sub-step 212. The complex image I.(x,y) is first Fourier 
transformed by the processor 105 in sub-step 520, using the FFT, thereby producing an 
image consisting of complex values. This image is separated in sub-step 530 into two 
separate images, those being a magnitude image containing the magnitudes of the 
complex values of the Fourier transform, and a phase image containing the phases of the 
complex values of the Fourier transform. In sub-step 560 a function is applied to the 
magnitude image, with the function being commutative within a constant to rotation and 
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scale. In the preferred implementation the magnitude image is multiplied by a ramp 
function to perform high-pass filtering of the magnitude image. In sub-step 570 an 
operator is applied to the phase image to take the second or higher derivative of the phase, 
which is a translation invariant. In the preferred implementation the Laplacian operator is 
used. 

Sub-step 214 continues to sub-step 580 where the modified magnitude image 
produced from sub-step 560, and the result of taking the Laplacian of the phase image 
produced from sub-step 570 are combinedthrough 

\F\ + iAV 2 <p 

(29) 

where \F\ is the modified magnitudes of the Fourier transform of the complex 
images I n {x,y); V> is the Laplacian of the phase image of the Fourier transform , and 
A is a scaling constant set to: 

if-nm<|FD/*. (30) 

The scaling constant A ensures that the recombined Fourier magnitude and phase 
information are roughly of equal magnitude. 

The result of the combining of the modified magnitude image and the result of 
taking the Laplacian of the phase image is then inverse Fourier transformed in sub-step 
590, thereby producing the representation T„(x,y) that is translation invariant in the 
spatial domain. 

Other translation invariants of the Fourier magnitude and phase may be used in 
place of sub-steps 560 and 580, such as: 

the modulus squared of the Fourier magnitude; 
the logarithm of the Fourier magnitude; 
the Laplacian of the logarithm of the Fourier transform; 



or 
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operators such as 



(31) 



and 



d 2 



10 



a* 2 dy 2 J'\dx^^)j^ (32) 

where the logarithm of a complex number is defined as 
IogF = log|F|+/0>. 

Preferably, the translation invariant is free of ISO- ambiguity with respect to 
rotation. This holds for all translation invariants listed above, except the modulus squared 
of the Fourier magnitude. 

Fig. 8 shows a more detailed flow diagram of the sub-step of performing in sub- 
step 216 the Fourier-Mellin correlation on the representations T^y) and T 2 ( X ,y) that 
are translation invariant in the spatial domain. In sub-step 630 each of the representations 
T^y) and T 2 ( x , y) is resampled to the log-polar domain. In order to resample to the 
log-polar domain, it is necessary to specify a resolution within the log-polar domain. If 
15 the original images /,(*,,) and are N pixels wide by M pixels high, hence the 

coordinate * varies between 0 and AM, while the ^-coordinate varies between 0 and M-l, 
then the centres of the representations T lix> y) and T 2 ( x , y) which are translation 
invariant in the spatial domain are located at ( c x ,c y ) = (floor(AV2),floor(M/2)). Log. 
polar resampling to an image having dimensions P pixels by Q pixels in log-polar space is 
20 performed relative to this centre. To avoid a singularity at the origin, it is necessary to 
ignore a disc of radius ^ pixels around the centres of the representations T^y) and 
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T 2 (x,y). While ignoring this disc, a point (xj) in the log-polar plane is detennined by 
interpolating the translation invariant image at the point (xy) as follows: 

y = Cy+ sinl£r^e« • ' (34) 

> s-i nun 

wherein 

P-l (35) 

and 

max {A/72, JV72} 

(36) 

denotes the maximum radius in the spatial domain that the log-polar image 
extends to. Common values of the constants r mia , PandQ are: 

0 

P = Q = (M + N)/2,znd (3?) 
r . =5. 

n,m (38) 
In sub-step 640 the processor 105 performs the Fourier transforms each of the 
resampled representations T t (x,y) and T 2 (x,y). Sub-step 640 is followed by sub-step 
650 where the processor 105 performs a complex conjugation on the second resampled 
representation T 2 (x,y). Both Fourier transforms are then normalised in sub-step 660 so 
that each has unit magnitude by dividing each complex element of each Fourier transform 
by the magnitude of the complex element. The normalised Fourier transforms are then 
multiplied in sub-step 670, and the result of the multiplication is then inverse Fourier 
transformed in sub-step 680. A phase correlation image results. 
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The above describes the prefer ,mpleme„.a«ons of.be preset disCo**, and 
* wiU be understood ^ mere „ . „ umberof modifcations ^ ^ ^ ^ ^ 

made memm without depaning ^ ^ ^ ^ ^ ^ ^ 

-~P 2,2 (Fig. 3, where ^ „ ^ ^ ^ ^ 

5 . s«,H mining sub-step 2,4 where mentations tha. are invariant t „ Nations fa fte 
spatial domain are formed. 

Method 200 has been deseribed in .em* of operations on iraag es r, M m 
Afe.) that have on. y one componen, Multi-component images, such as colour images 
and images from mutti-apeetia! devices' can a,so be regis^ witb respect ,o each other 
■0 using method 200. Tftis is achieved b y fonning a smgie component image from each of 
the mttiti-componen, images mrongh some a.gebraic combination of the components, a^ 
« me two sing,, eomponen, images ^ ^ ^ ^ 

Aitemativetv « may be achieved b y Bering each component of me flrs, mu,ti- 
componen, image against each component of the other mu,«-compone„t image 
5 separately. 

b. the come* of tins specification, me won, "compristag" means "mCnding 
principal bu, no, „ecessari ly so,e ly " or "beving" or "mCnding", and no. -consisting 
on, y of. Variations of me word "comprising., such as "comprise- and "comprise,. bave 
correspondingly varied meanings. 
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The claims defining the invention are as follows: 

1. A method of determining at least rotation and scale parameters of a transformation 
relating two images, said method comprising the steps of: 

5 forming a spatial domain representation of each of said images that is invariant 

to translation of said image; 

performing correlation in the log-polar domain between said representations; 
detecting a magnitude peak in said correlation; and 

determining said rotation and scale parameters from the position of said 

t 

10 magnitude peak. 

2. A method as claimed in claim 1 wherein the step of forming said representation of 
said images comprises the sub-steps of, for each said image: 

performing a Fourier transform of said image to form a Fourier transformed 

15 image; 

performing a function on the magnitude component of said Fourier transformed 
image to form an altered Fourier transformed image, said function being commutative 
within a constant to rotation and scale; and 

performing an inverse Fourier transform on said altered Fourier transformed 
20 image to form said representation. 

.3. A method as claimed in claim 1 wherein the step of forming said representation of 
said images comprises the sub-steps of, for each said image: 

performing a Fourier transfonn of said image to form a Fourier transformed 

25 image; 



« 
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performing a function on the magnitude component of said Fourier transformed 
image to form an altered Fourier magnitude image, said function being commutative 
within a constant to rotation and scale; 

taking the second or higher derivatives of the phase component of said Fourier 
transformed image to form an altered Fourier phase image; 

combining said altered Fourier magnitude and altered Fourier phase images to 
form an altered Fourier transformed image; and 

performing an inverse Fourier transform on said altered Fourier transformed 
image to form said representation. 



4. A method as claimed in claim. 3 wherein said altered Fourier phase image is formed 
by applying the Laplacian operator to said phase component of said Fourier transformed 
image. 

5. A method as claimed in claim 3 or 4 wherein said altered Fourier magnitude and 
altered Fourier phase images are combined by using said altered Fourier magnitude image 
as a real part of said altered Fourier transformed image, and using said altered Fourier 
phase image as an imaginary part of said altered Fourier transformed image. 

6. A method of determining at least rotation and scale parameters of a transformation 
relating two images, said method comprising the steps of: 

forming a multi-channel function of each of said images by applying an operator 
to said images, said operator being commutative within a constant to rotation and scale; 

forming a representation of each of said multi-channel functions that is invariant 
to translation of said multi-channel function; 
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performing correlation in the log-polar domain between said representations; 
detecting a magnitude peak in said correlation; and 

determining said rotation and scale parameters from the position of said 
magnitude peak. 

7. A method as claimed in claim 6 wherein the step of forming said multi-channel 
functions comprises the sub-steps of, for each image: 

convolving said image with a complex, kernel function; and 
multiplying said image with the result of the convolution step, wherein said 
complex kernel function has the Fourier transform of: 

AXh,v) = - 

|w + iv| 

8. A method as claimed in claim 6 wherein the step of forming said multi-channel 
functions comprises the sub-steps of, for each image: 

convolving said image with a complex kernel function; and 
multiplying said image with the result of the convolution step, wherein said 
complex kernel function has the Fourier transform of: 



9. A method as claimed in claim 6 wherein the step of forming said multi-channel 
functions comprises, for each image: 

applying an energy operator to said image to foim said multi-channel function, 
where said energy operator is described by 
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£[/] = /D 2 /-(Z>/) 2 , 
wherein D is the derivative operator. 

10. A method as claimed in claim 6 wherein the step of forming said multi-channel 
functions comprises, for each image: 

applying a uni-modular energy operator to said image to form said multi-channel 
function, where said uni-modular energy operator is described by 

EV] = ID«I-{D'I)\ 

wherein D' is the uni-modular derivative operator. 

11. A method as claimed in claim 9 or 10 wherein the step of forming said multi- 
channel functions comprises the further sub-step of: 

normalising the result of the applying step. 

12. A method as claimed in claim 9 or 10 wherein the step of forming said multi- 
channel functions comprises the further sub-step of: 

multiplying said image with the result of the applying step. 

13. A method as claimed in claim 9 or 10 wherein the step of forming said multi- 
channel functions comprises the further sub-steps of: 

normalising the result of the applying step; and 

multiplying said image with the result of the normalising step. 
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14. A method as claimed in any one of claims claim 6 to 13 wherein said 
representations are in the spatial domain. 

15. A method as claimed in any one of claims claim 1 to 14 wherein said correlation is 
5 the Fourier-Mellin correlation. 

16. An apparatus for determining at least rotation and scale parameters of a 
transformation relating two images, said apparatus comprising: 

means for forming a spatial domain representation of each of said images that is 
10 invariant to translation of said image; 

means for performing correlation in the log-polar domain between said 
representations; 

means for detecting a magnitude peak in said correlation; and 
means for determining said rotation and scale parameters from the position of 
15 said magnitude peak. 

17. An apparatus for determining at least rotation and scale .parameters of a 
transformation relating two images, said apparatus comprising: 

means for forming a multi-channel function of each of said images by applying 
20 an operator to said images, said operator being commutative within a constant to rotation 
and scale; 

means for forming a representation of each of said multi-channel functions that 
is invariant to translation of said multi-channel function; 

means for performing correlation in the log-polar domain between said 
25 representations; 
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means for detecting a magnitude peak in said correlation; and 
means for determining said rotation and scaje parameters from the position of 
said magnitude peak. 

18. A program stored on a memory medium for determining at least rotation and scale 
parameters of a transformation relating two images, said program comprising: 

code for forming a spatial domain representation of each of said images that is 
invariant to translation of said image; 

code for performing correlation in the log-polar domain between said 
representations; 

code for detecting a magnitude peak in said correlation; and 
code for determining said rotation and scale parameters from the position of said 
magnitude peak. 

1 9. A program stored on a memory medium for determining at least rotation and scale 
parameters of a transformation relating two images, said program comprising: 

code for forming a multi-channel function of each of said images by applying an 
operator to said images, said operator being commutative within a constant to rotation and 
scale; 

code for forming a representation of each of said multi-channel functions that is 
invariant to translation of said multi-channel function; 

code for performing correlation in the log-polar domain between said 
representations; 

code for detecting a magnitude peak in said correlation; and 
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code for determining said rotation and scale parameters from the position of said 
magnitude peak. 



20. A mchod of defining a, ,eas. rotation and scale parameters of a transformation 
relating two images, said method being subs.an.iaHy as described herein with reference .o 
the accompanying drawings. 

DATED this Eighth Day of JULY 2003 
CANON KABUSHIKI KAISHA 
Patent Attorneys for the Applicant 
SPRUSON&FERGUSON 
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